Journals
  Publication Years
  Keywords
Search within results Open Search
Please wait a minute...
For Selected: Toggle Thumbnails
Sentence embedding optimization based on manifold learning
Mingyue WU, Dong ZHOU, Wenyu ZHAO, Wei QU
Journal of Computer Applications    2023, 43 (10): 3062-3069.   DOI: 10.11772/j.issn.1001-9081.2022091449
Abstract196)   HTML8)    PDF (1411KB)(125)       Save

As one of the core technologies of natural language processing, sentence embedding affects the quality and performance of natural language processing system. However, the existing methods are unable to infer the global semantic relationship between sentences efficiently, which leads to the fact that the semantic similarity measurement of sentences in Euclidean space still has some problems. To address the issue, a sentence embedding optimization method based on manifold learning was proposed. In the method, Local Linear Embedding (LLE) was used to perform double weighted local linear combinations to the sentences and their semantically similar sentences, thereby preserving the local geometric information between sentences and providing helps to the inference of the global geometric information. As a result, the semantic similarity of sentences in Euclidean space was closer to the real semantics of humans. Experimental results on seven text semantic similarity tasks show that the proposed method has the average Spearman’s Rank Correlation Coefficient, (SRCC) improved by 1.21 percentage points compared with the contrastive learning-based method SimCSE (Simple Contrastive learning of Sentence Embeddings). In addition, the proposed method was applied to mainstream pre-trained models. The results show that compared to the original pre-trained models, the models optimized by the proposed method have the average SRCC improved by 3.32 to 7.70 percentage points.

Table and Figures | Reference | Related Articles | Metrics